The present disclosure relates generally to cloud computing, and more particularly to a more efficient and scalable method for utilizing the scarce resources in a cloud computing system.
Cloud computing services can provide computational capacity, data access, networking/routing and storage services via a large pool of shared resources operated by a cloud computing provider. Because the computing resources are delivered over a network, cloud computing is location-independent computing, with all resources being provided to end-users on demand with control of the physical resources separated from control of the computing resources.
Originally the term cloud came from a diagram that contained a cloud-like shape to contain the services that afforded computing power that was harnessed to get work done. Much like the electrical power we receive each day, cloud computing is a model for enabling access to a shared collection of computing resources—networks for transfer, servers for storage, and applications or services for completing work. More specifically, the term “cloud computing” describes a consumption and delivery model for IT services based on the Internet, and it typically involves over-the-Internet provisioning of dynamically scalable and often virtualized resources. This frequently takes the form of web-based tools or applications that users can access and use through a web browser as if it was a program installed locally on their own computer. Details are abstracted from consumers, who no longer have need for expertise in, or control over, the technology infrastructure “in the cloud” that supports them. Most cloud computing infrastructures consist of services delivered through common centers and built on servers. Clouds often appear as single points of access for consumers' computing needs, and do not require end-user knowledge of the physical location and configuration of the system that delivers the services.
The utility model of cloud computing is useful because many of the computers in place in data centers today are underutilized in computing power and networking bandwidth. People may briefly need a large amount of computing capacity to complete a computation for example, but may not need the computing power once the computation is done. The cloud computing utility model provides computing resources on an on-demand basis with the flexibility to bring it up or down through automation or with little intervention.
As a result of the utility model of cloud computing, there are a number of aspects of cloud-based systems that can present challenges to existing application infrastructure. First, clouds should enable self-service, so that users can provision servers and networks with little human intervention. Second, network access is necessary. Because computational resources are delivered over the network, the individual service endpoints need to be network-addressable over standard protocols and through standardized mechanisms. Third, multi-tenancy. Clouds are designed to serve multiple consumers according to demand, and it is important that resources be shared fairly and that individual users not suffer performance degradation. Fourth, elasticity. Clouds are designed for rapid creation and destruction of computing resources, typically based upon virtual containers. Provisioning these different types of resources must be rapid and scale up or down based on need. Further, the cloud itself as well as applications that use cloud computing resources must be prepared for impermanent, fungible resources; application or cloud state must be explicitly managed because there is no guaranteed permanence of the infrastructure. Fifth, clouds typically provide metered or measured service—like utilities that are paid for by the hour, clouds should optimize resource use and control it for the level of service or type of servers such as storage or processing.
Cloud computing offers different service models depending on the capabilities a consumer may require, including SaaS, PaaS, and IaaS-style clouds. SaaS (Software as a Service) clouds provide the users the ability to use software over the network and on a distributed basis. SaaS clouds typically do not expose any of the underlying cloud infrastructure to the user. PaaS (Platform as a Service) clouds provide users the ability to deploy applications through a programming language or tools supported by the cloud platform provider. Users interact with the cloud through standardized APIs, but the actual cloud mechanisms are abstracted away. Finally, IaaS (Infrastructure as a Service) clouds provide computer resources that mimic physical resources, such as computer instances, network connections, and storage devices. The actual scaling of the instances may be hidden from the developer, but users are required to control the scaling infrastructure.
One way in which different cloud computing systems may differ from each other is in how they deal with control of the underlying hardware and privacy of data. The different approaches are sometimes referred to a “public clouds,” “private clouds,” “hybrid clouds,” and “multi-vendor clouds.” A public cloud has an infrastructure that is available to the general public or a large industry group and is likely owned by a cloud services company. A private cloud operates for a single organization, but can be managed on-premise or off-premise. A hybrid cloud can be a deployment model, as a composition of both public and private clouds, or a hybrid model for cloud computing may involve both virtual and physical servers. A multi-vendor cloud is a hybrid cloud that may involve multiple public clouds, multiple private clouds, or some mixture.
Because the flow of services provided by the cloud is not directly under the control of the cloud computing provider, cloud computing requires the rapid and dynamic creation and destruction of computational units, frequently realized as virtualized resources. Maintaining the reliable flow and delivery of dynamically changing computational resources on top of a pool of limited and less-reliable physical servers provides unique challenges.
The typical way in which resources are allocated uses a “scheduler,” a controller that performs a function similar to a scheduler in an operating system. An operating system scheduler balances the competing demands for resources and allocates access to scarce resources such as CPU time, physical memory, virtual memory, disk accesses, disk space, or processes. In a cloud computing context, the scheduler allocates resources such as network capacity, disk capacity, disk activity on hosts, physical CPU time on hosts, and latency to identify a “best” physical host on which to situate a virtual machine (or other virtual resource). Sometimes the schedulers use randomness to achieve a probabilistic spread of load across the physical resources. Other schedulers use a pre-set spreading rule that achieves a particular desired distribution of work. A third type of schedulers is adaptive, reacting to changes and modifying allocation rules.
Nevertheless, there are two major factors that distinguish the cloud scheduler case from the operating system scheduler case. First is the availability of “intelligence” to sense local conditions and make location-specific optimization decisions. Operating system schedulers cannot have a particular block of memory “decide” about its most efficient utilization, so a central command-and-control scheme is the only viable option. Second is the scale of cloud-based systems. Cloud schedulers must regularly deal with numbers of resources and competing demands that outstrip regular CPUs and multiprocessor systems and the largest cloud systems must deal with numbers of resources much higher than even supercomputers. Because of the size of the optimization problem, it is difficult to reach optimal (or near-optimal) decisions when faced with substantial load.
Further complicating the issue, most cloud systems assume a relatively homogenous pool of underlying computing resources, on top of which a homogenous pool of virtualized resources is instantiated. This is a useful abstraction, but it ignores the underlying differences in hardware. Even when the hardware and virtualized environments are identical, they still vary relative to relative latency, load due to multi-tenancy, disk activity, and differences in performance of the underlying hardware. Further, private clouds especially are created from repurposed servers from other projects or from a pool of unused servers. The hardware components of these servers can vary significantly, causing instance performance variances based on the capabilities of the host compute node for a particular instance.
The differing capabilities and challenges of cloud computing systems make it possible for a local-intelligence-directed system of resource allocation could be more effective than a centralized scheduler. One method of distributed allocation of resources is using market mechanisms, in which different items have a “cost” and an expected utility. Market-based mechanisms can produce globally optimal resource allocations under the correct conditions, even when each actor in the system uses only local decision criteria and local optimization functions.
Accordingly, it is desirable to provide a better-functioning cloud computing system with superior operational capabilities by using local intelligence to create a market-based resource allocation scheme for cloud computing systems.